Skip to content

Proposal: Lotus / Filecoin RPC and Library Audit #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 9, 2021

Conversation

rvagg
Copy link
Contributor

@rvagg rvagg commented Mar 15, 2021

Sudo is currently working on store-my-text and it's hitting all of the hurty-parts of small-deal making, but it's also raising lots of questions about, and finding problematic issues with, the JavaScript API landscape and the RPC API itself. So this project is intended as a broader RPC API audit as an extension of that work (or replacement of it).

@github-actions github-actions bot requested a review from jacobheun March 15, 2021 07:17
@vasco-santos vasco-santos self-requested a review March 15, 2021 19:50
Copy link
Contributor

@vasco-santos vasco-santos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good to me.

One of the problems I have seen after playing with filecoin-shipyard/js-lotus-client is that the type definitions generated from go do not have into consideration parameters optionality. This basically means that a typescript project needs to provide all the parameters, even if most (of all of them) are optional, in order to have tsc running. The way the schemas are generated might mean that we do not have a lot of flexibility as the API is basically inferred by generating typescript declarations from lotus api source code.


See list of outputs above.

#### What does success look like?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add some quantitative success metrics for this work? Maybe time to build something successful using js libs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually probably a chain of success metrics:

  • Ability to rate/sort js lib proposals
  • high-value js lib proposals identified and chosen to work on

Describe any choices and uncertainty in this scope estimate. (E.g. Uncertainty in the scope until design work is complete, low uncertainty in execution thereafter.)
-->

S / M
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like time-boxing this research is important to ensure we also get to execution on the highest priority items... maybe do a rough timeline here?

@rvagg
Copy link
Contributor Author

rvagg commented Mar 22, 2021

I've tried to clarify scope here a little better, and fill out some of the additional missing sections.

This project is doing a lot of mapping-of-the-unknown, but it does need to be kept to the minimum required. We currently think we have a decent picture of the landscape so the scope is defined accordingly--although we need to be prepared to discover entirely new universes (I stumbled on the glifio JS stack by accident just prior to writing this proposal!). Where I'm keeping this tight is in the maturity of the output. We're not producing polished docs, we're not even producing docs except where modifications might be trivial activities. We're producing checklists, and have already started that process. We'll scope further work later but we just need to collect this information by exploring and very basic experimenting.

I've sized it at a Small, but that's for 3-5 FTEs, which is much more than we're working with on Sudo at the moment so I want to set that expectation up front.

@anorth @jacobheun I would appreciate some EM eyes to critique my attempts to define a sense of scope.

@jacobheun
Copy link
Contributor

except where modifications might be trivial activities

I would avoid this regardless of triviality to prevent frequent yet subtle derailments. Reading through this the output seems to best fit a report on the current state of JS clients, and recommendations (stack ranked) for improvements.

Based on the information here it seems that stakeholder interviews should be the first to happen, which would then inform the existing api/client evaluation: Based on what we learned from the stakeholders, what are the libraries we should be focusing our analysis on?

It seems like this is the general path being proposed here, I just want to verify we're not proposing these may be done in parallel. The advantage of doing these in sequence is that we can reduce the footprint of the api we're evaluating and then focus investigation on verifying the observations from the interviews. During validation we may find additional issues that weren't previously raised, but those can be added to the findings of the final report. Overall the scope looks good, but we should ensure to phase it, to avoid splitting focus over too many things.

@mikeal mikeal added ease:high Ease rating is 6 or above. impact:high Impact rating is 6 or above. labels Mar 25, 2021
@anorth
Copy link
Contributor

anorth commented Mar 26, 2021

Scope for this project seems appropriately limited. I wouldn't want that to also limit the scope of recommendations that come out of this, though. E.g. maybe the Lotus RPC API that serves node operators and miners isn't the right abstraction, or transport, or contract-to-impose-on-Lotus for client APIs, and work on the libraries that try to solve that impedance mismatch entrenching a long-term problem?

@rvagg
Copy link
Contributor Author

rvagg commented Mar 29, 2021

Updated the "done" to say that the goal is to produce a "brief report" covering the state of the world for these libraries and the Lotus RPC API from the library perspective. It should be quite terse and short and aim for maximal information in minimal words and be useful for scoping and prioritising future work and may feed into documentation efforts.

@jacobheun jacobheun merged commit 591e59f into main Jun 9, 2021
@jacobheun jacobheun deleted the rvagg/lotus-api-and-library-audit branch June 9, 2021 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ease:high Ease rating is 6 or above. impact:high Impact rating is 6 or above.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants